Optimizing INTRA/INTER Coding Mode Decisions

In this paper, we present a new approach for INTRA/INTER mode decisions. In the current MPEG verification model (VM), the coding mode for each macroblock is selected by comparing the energy of predictive residuals. Specifically speaking, the INTRA/INTER decision is determined by comparing the variance of the macroblock pixels against the predictive residuals. However, the coding mode selected by the VM criteria does not result in the optimal coding performance. We observe that in deciding which of the various coding modes is best, one should not only consider spending bits for coding the prediction residuals but also consider spending bits for coding motion vectors. The simulation results show that our method provides better SNR while using the same amount of bits.